Skip to content

feat(tools): queue hosted-key tool calls instead of failing with 429#4416

Draft
TheodoreSpeaks wants to merge 1 commit intostagingfrom
feat/queued-hosted-key
Draft

feat(tools): queue hosted-key tool calls instead of failing with 429#4416
TheodoreSpeaks wants to merge 1 commit intostagingfrom
feat/queued-hosted-key

Conversation

@TheodoreSpeaks
Copy link
Copy Markdown
Collaborator

Summary

  • Hosted-key tool calls now wait for the per-workspace token bucket to refill instead of returning 429 immediately. Behavior is on for every hosted-key call; BYOK paths short-circuit before acquisition and are unaffected.
  • Wait is capped at 5 minutes (well under the 90-min Trigger.dev container ceiling). If the cap is exceeded, falls back to today's 429 result.
  • Per-workspace+provider FIFO lock (Redis SETNX, key hosted-queue:{provider}:{workspaceId}) serializes callers within a workspace so the bucket drains predictably. Different workspaces never block each other. No-op when Redis is unavailable.
  • On terminal upstream 429 (after the existing 3-attempt exponential backoff), executeWithRetry re-enters the queue, re-acquires a fresh hosted key, and retries once.
  • New telemetry: platform.hosted_key.queue_waited and platform.hosted_key.queue_wait_exceeded.

Type of Change

  • New feature

Testing

  • 17 hosted-key rate-limiter tests pass (4 reworked, 2 new for wait-then-succeed)
  • 23 broader rate-limiter tests pass; 73 tools/index tests pass
  • bun run lint clean
  • bun run check:api-validation:strict passes

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel
Copy link
Copy Markdown

vercel Bot commented May 3, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped May 3, 2026 1:42am

Request Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant